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ABSTRACT 

This study investigates the influence of sensory and cognitive affordances on the usability of mobile devices for 
multimedia language learning applications. An audio-based learning application - the ‘Vowel Trainer’ (audio-based 
speech app), developed by University College London was chosen, against a comparison, text and picture -based language 
learning application - ‘Learn English for Taxi Drivers’ from the British Academy (app based on vocabulary, grammar). 
Impressions of the two applications were assessed on two different devices that have virtually the same interface, but 
differ in physical size: the iPhone and the iPad. A mixed design was chosen, with device type (iPad vs. iPhone) as 
between groups factor and language application type (audio vs. video) as a within groups factor. Assessments of sensory 
and cognitive affordances were made, along with measurement of learner preferences of each application and their 
suitability for each device. A preliminary (N=22) dataset was analysed using a mixed design MANOVA and the 
implications of the results for developing design guidelines for spoken language learning applications are discussed. 
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1. INTRODUCTION 

There use of mobile technologies for language learning is not new and there is an abundance of research on 
learning outcomes of these newer language-learning technologies incorporating multimedia (e.g. Guerrero, 
Ochoa, & Collazos, 2010). However concerted efforts to incorporate usability principles into designing 
language-learning software for mobile devices have been relative scant. A large body of research so far has 
been done with visually-based applications in mind (e.g. Chinnery, 2006; Fisher et al., 2009; Kukulska- 
Hulme, 2005; Kukulska-Hulme & Shield, 2008), but the generalisability to audio-based applications has not 
been directly empirically tested. Given the suggestion that some of these devices might actually have a better 
suitability for audio and speech applications (Uther, 2002; Uther et al., 2007; Uther, Zipitria, Uther, & Singh, 
2005), and the possibility that mobile devices may offer a more portable and personalised learning solution 
(Kukulska-Hulme & Shield, 2008), it is vital that these new technologies are considered and evaluated 
appropriately for their effectiveness in relation to language learning applications and the requirements that 
those kinds of applications might have. 

A useful and interesting line of investigation is the use of ‘affordances’ to guide what might make a 
usable application. In the usability field, the development (and popularisation) of the term ‘affordances’ came 
with the work of Norman (1988). Norman suggests that there are both perceived and actual properties of an 
object that can lend itself to affordances (e.g. a chair can afford the act of sitting, but the chair can also be 
carried (McGrenere & Ho, 2000)). In more recent years, distinction between perceived and ‘real’ affor dance 
has resulted in a useful further development in what could be termed a ‘multiple abstractions’ view of 
affordance. A framework developed by Hartson (2003) is useful, which specifies kinds of affordances: 
physical, sensory, cognitive and functional. This study focuses on the influence of sensory and cognitive 
affordances. A sensory affordance, Hartson argues, is having a font size that is large enough to read the label 
clearly. One could also specify further that sensory affordances could be a product of the software (as in the 
font size of the actual label), but these may also be influenced by the user’s perception of the quality. 
Cognitive affordances on the other hand are design features that assist the user in identifying how a particular 
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object could be used. For example, smart phones have buttons and physical interfaces that signify that this is 
a device that is for speaking and listening into. Hartson’s framework is a useful one for evaluating 
affordances of mobile language learning applications, particularly as for example audio quality might 
obviously affect spoken language learning outcomes. 

Of course, affordances are generally speaking very difficult to test within controlled experiments because 
hardware design often determines a priori whether the affordance is there or not (for example, a very small 
screen will simply not afford to the task of reading as well as a large screen, despite the best attempts of a 
software designer). As engineers embark on design of hardware, consideration is generally restricted to the 
physical affordance level, with consideration of cognitive or sensory affordance left to the task of software 
designers. The difficulty in investigating the role of cognitive or sensory affordances for applications such as 
language learning technology is that there are a myriad of different kinds of devices that may be used for 
language learning (iPods, iPads, smartphones, PCs, tablets, etc.). Each have different physical constraints and 
it is difficult to match these in a controlled way to inform good practice in user design. However, with the 
development of the iPad and iPhone, there is a unique opportunity to study affordances for use of multimedia 
in language learning. With both types of devices, there are similar if not identical video and audio capabilities 
(audio quality is identical, although video quality is superior). Physical interaction styles - screen layout and 
the physical interface (navigation button and swipe) are also identical in both cases. 

Studying cognitive and sensory affordances can provide useful information that would inform future 
design of spoken language learning technologies. As it has been hypothesised (Uther, 2002; Uther et al., 
2005) that mobile phones have an affordance for spoken language, it would be useful to compare spoken 
language learning applications in a mobile phone that has a virtually identical counterpart in a larger device. 
From the ‘cognitive affordance’ point of view, one could argue that the very nature of a phone signals 
functions for speaking and listening to. Such a comparison would also provide a useful framework for 
evaluation of these technologies that can assist practitioners conducting usability studies for industry. In this 
study, two software applications with two different kinds of multimedia (audio-rich vs. graphics-rich) are 
compared: an audio-intensive language learning program (UCL Vowel Trainer) and a more text- and picture- 
rich language learning program, focused on grammar (‘Learn English for Taxi Drivers’ by the British 
Academy). Both software applications are tested on both types of devices (iPhone and iPad). In terms of 
sensory affordances, it would be predicted that both devices fare equally well in terms of audio quality. 
However, video content may be rated more highly on the iPad due to its superior video bit rate and larger 
screen size. In terms of cognitive affordances, it could be predicted that iPads might afford better for the 
video-based application as it has a larger screen and iPhones may afford for the audio language learning 
application as it is seen as better for listening. Equally, one could also plausibly predict that both types of 
applications might afford equally well to the iPad because the iPad might be seen by users as a better 
‘educational’ tool. 


2. EXPERIMENT 

A small-scale study (N=22) was conducted on 12 native speakers of English and 10 non -native speakers of 
English. Methods and preliminary results are briefly described below. 

2.1 Method 

2.1.1 Participants 

Participants were aged 18-26 years old (mean=19, SD=18) and were recruited from the University of Surrey. 
Participants were studying Psychology, with the exception of two students studying Engineering. Psychology 
students received course credit for their participation. None suffered hearing or visual impairments. 

2.1.2 Procedure 

Participants were given informed consent, and filled questions relating to their demographic and language 
background. They were then given a selection of standard music, audio book and video samples (approx. 15 
seconds each) to listen to on both iPad v.2 and iPhone v.4. They listened to the samples in a quiet room. 
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using a standard pair of Sennheiser headphones set to a comfortable hearing level. They completed a set of 
questionnaires throughout the testing asking them to rate the sound and picture (where applicable) quality of 
the samples as they listened to each. The participants then were given a demonstration and opportunity to 
have a short trial of the two language learning software applications. They were asked to rate the software on 
both devices for sound and picture quality. They were also asked to rate the suitability of each software for 
each device and how likely it would be that they would procure such an application if they were a non -native 
speaker. The participants also rated usability of software applications using the Questionnaire for User 
Interface Satisfaction (QUIS). Preliminary results of the affordance and suitability data are only presented in 
this paper due to time and space constraints. 

2.2 Results 

2.2.1 Sensory Affordance Questions 

A comparison of participant-rated sound quality showed no significant differences between the iPad and 
iPhone devices. For the video sample, participants rated both picture and sound quality as higher in the iPad 
compared to the iPhone (Fi ^o = 8.391, p<0.05 and Fi, 20=20.792, p<0.05 respectively), see Figure 1 below. 
There were no differences in ratings between native and non-native speakers. 
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Figure 1. Mean quality ratings of video clip sample on iphone and ipad. Error bars show standard deviation. 

2.2.2 Cognitive Affordance Questions 

Analysis of sound quality for the Vowel Trainer across devices showed that participants rated sound quality 
as better on the iPad compared to the iPhone (Fi 2o=4.611, p<0.05, see Table 1 below). Analysis of the sound 
and picture quality of the Learn English application showed that there were no significant differences 
between the rated sound quality on the iPad vs. iPhone. However, participants did rate the picture quality as 
superior on the iPad for the Learn English application (Fi 2o=4.638, p<0.05, see Table 1). 


Table 1. Comparison of participant ratings of sound and picture quality on the two software applications on both devices. 

Mean values are given, with standard deviations in brackets. 




iphone 


Ipad 


Vowel Trainer 

Learn English 

Vowel Trainer 

Learn English 

Sound quality 

4.81 (1.53) 

5.77(1.19) 

5.23(1.54) 

5.91 (1.15) 

Picture quality 

- 

4.90(1.15) 

- 

5.54(1.26) 


Analysis of rated suitability and likelihood of future use showed similar patterns of results. There were no 
overall main effects of device nor software. However, there was a significant interaction between software 
and device (Fi 2o=7.580, p<0.05 for suitability and Fi 2o=5.649, p<0.05 for likelihood), with participants 
rating that they felt the suitability and likelihood of future use was rated more positive for the iPhone 
compared with the iPad for the vowel trainer, whereas the converse was true for the Learn English software, 
see Figure 2 below. 
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Figure 2. Participant-rated suitability of different language learning applications on ipad and iphone. 


3. CONCLUSION 

The preliminary results of the study show that there appears to be little difference in the subjective perception 
of sound quality between the two devices for audio content. When evaluating video content, picture quality 
was perceived to be superior on the iPad, which mirrors the physical differences between the devices. 
However, for video samples, audio quality was also perceived to be superior in the iPad, despite there being 
no differences in audio quality. At a user level, it appears the iPad has more of a superior rating for both 
picture and audio quality. Interesting, when looking at whether these sensory affordances affect cognitive 
perceptions of suitability of the language learning applications, users appear to disregard these sensory 
perceptions when it comes to audio-based applications when assessing suitability and likelihood of future 
use. Further research and analysis needs to focus on whether there are differences in perceptions between the 
two types of language learning software or other contextual factors that underpin these cognitive perceptions. 
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